You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Which issue or RFC does this PR close? Implements the first bounded Memory write-gate slice of RFC #1770. Refs #1770, #1742, and #1745. This PR does not close the broader implementation or evaluation issues. # Rationale for this change RFC #1770 requires an explicit policy, privacy, fallback, observation, and promotion contract before additional domain consumers adopt decision models. Memory write evidence sufficiency is the narrowest existing boundary: the gate may produce a visible HOLD, but Memory retains authority over commits and revisions. The design borrows only bounded governance principles from Hermes/JEV research: explicit modes, deterministic local rules, safe fallback, and observable outcomes. It does not copy Hermes storage, lifecycle, prompts, thresholds, or host-specific implementation. # What changes are included in this PR? - Adds runtime-independent DecisionPolicy, DecisionAssessment, and DecisionObservation contracts. A fallback is always unadjudicated. - Adds an opt-in Memory evidence-sufficiency gate with disabled -> shadow -> advisory -> enforcing behavior. Runtime configuration defaults to shadow; enforcing is an explicit operator choice. - Binds the gate's hold direction and confidence threshold into the effective immutable policy version, and declares labeled calibration, measured friction/cost/latency, and rollback as promotion criteria. - Applies mode semantics and a private observation to deterministic evidence-projection rejections as well as model decisions. Existing custom gates without the optional preflight contract retain conservative legacy behavior. - Enforces the privacy boundary on every gate construction path. Raw candidate/evidence text is evaluated only when local_only is selected and the resolved backend is explicitly loopback (localhost, 127.0.0.1, or ::1). Unknown, LAN, wildcard, and hosted endpoints fail open without receiving raw content. no_external_call, references_only, and unsupported hosted_redacted do not make a content call. - Emits bounded structured logs and a private, scope-isolated durable pc_decision_observations ledger. Observations retain policy/version/mode, stable references, counts, verdict/action, privacy boundary/outcome, and fallback codes only; they never store candidate text, evidence text, model rationale, or exception details. - Adds exact observation replay idempotency, same-ID conflict rejection, and optional decision_observation_retention_days cleanup limited to the current Scope. The default is no automatic deletion. - Completes MySQL schema/index budgeting and bilingual obloader recovery lists for the new table. # Are there any user-facing changes? No public HTTP/OpenAPI endpoint, CLI command, dashboard, export surface, hosted-redacted transport, replay/rescore job, query rewrite, or additional decision-model consumer is added. The Memory write gate remains disabled unless explicitly enabled. When enabled, its runtime default is shadow and its privacy default is no_external_call; it only records a no-call observation and preserves the write. An operator must explicitly select local_only, configure a loopback backend, calibrate the policy, and choose enforcing before raw content is evaluated and a write can be held. # How was this change tested? - uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_contract.py tests/builtin/runtime/test_memory_write_gate_paths.py tests/builtin/runtime/test_decision_config.py tests/builtin/runtime/test_decision_composition.py tests/builtin/runtime/test_decision_policy.py tests/test_server_logging.py tests/builtin/persistence/test_decision_observation_repository.py tests/builtin/persistence/test_mysql_schema.py -q (129 passed) - uv run --no-sync ruff check <16 changed files> - uv run --no-sync ruff format --check <16 changed files> - uv run --no-sync ty check --python-version 3.11 <14 changed Python files> - git diff --check Validation limitation: the broader tests/builtin/runtime -q run consistently stops in the unchanged Windows spawned-worker diagnostic case expecting worker_crash but observing processing_failed (197 passed, 1 failed before --maxfail=1). This PR does not modify that worker or diagnostic path; the focused decision, persistence, logging, and composition suites above were run after the final changes. # AI usage statement Implemented with OpenAI Codex in the Codex desktop app from RFC #1770 and repository-local inspection. Hermes/JEV was consulted only as architectural prior art; no Hermes source or implementation structure was copied.
Follow-up
Promotion to enforcing is not part of this PR. The replay/re-evaluation, calibration, held-out evaluation, and promotion decision needed to consider it are tracked in #1930.
Add the internal DecisionPolicy, DecisionAssessment, and DecisionObservation vocabulary from RFC 1770. Route the Memory write gate through the shared assessment mapping before preserving its existing ACCEPT/FLAG/HOLD contract.
Constraint: no public HTTP/OpenAPI surface, persistence schema, or hosted-provider configuration is added.
Rejected: copying Hermes implementation code or plugin structure; only the RFC-level governance semantics are mirrored.
Scope-risk: limited to runtime decision vocabulary and Memory write gate assessment mapping.
Tested: UV_PROJECT_ENVIRONMENT=.venv312 uv run --python 3.12 --no-sync pytest tests/builtin/runtime/test_decision_policy.py tests/builtin/runtime/test_memory_write_gate_contract.py::test_memory_write_decision_policy_keeps_fallback_unadjudicated tests/builtin/runtime/test_memory_write_gate_contract.py::test_memory_write_decision_policy_uses_configured_hold_direction -q
Tested: UV_PROJECT_ENVIRONMENT=.venv312 uv run --python 3.12 --no-sync ruff check src/powercontext/builtin/runtime/decision_policy.py src/powercontext/builtin/runtime/memory_write_gate.py src/powercontext/builtin/runtime/__init__.py tests/builtin/runtime/test_decision_policy.py tests/builtin/runtime/test_memory_write_gate_contract.py
Tested: UV_PROJECT_ENVIRONMENT=.venv312 uv run --python 3.12 --no-sync ruff format --check src/powercontext/builtin/runtime/decision_policy.py src/powercontext/builtin/runtime/memory_write_gate.py src/powercontext/builtin/runtime/__init__.py tests/builtin/runtime/test_decision_policy.py tests/builtin/runtime/test_memory_write_gate_contract.py
Not-tested: full Memory write gate async contract on this Windows host hangs in CPython Proactor event-loop socketpair setup; default uv Python 3.14 currently fails tests/conftest.py typing import; ty check cannot resolve pydantic/pytest in this environment.
Co-authored-by: OmX <omx@oh-my-codex.dev>
Apply the RFC 1770 policy mode, privacy-boundary, and observation rules to the existing Memory write gate. Default configuration now prevents content egress, while local-only evaluation keeps the established opt-in enforcement path.
Constraint: no HTTP/OpenAPI contract, persistence schema, durable observation ledger, hosted sanitizer, or provider integration is added.
Scope-risk: limited to Memory write-gate configuration, decisions, privacy-safe observation logs, and request attribution.
Tested: uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_contract.py -q (26 passed)
Tested: uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_contract.py tests/builtin/runtime/test_memory_write_gate_paths.py -q (46 passed)
Tested: uv run --no-sync ruff check <11 changed files>; uv run --no-sync ruff format --check <11 changed files>; uv run --no-sync ty check --python-version 3.11 <11 changed files>
Not-tested: a final Windows full-runtime retry was stopped after exceeding the prior 175s run time. The only earlier full-suite failure was reproduced against the exact b1d544b baseline in the spawned-worker diagnostics path, unrelated to these files.
The reason will be displayed to describe this comment to others. Learn more.
Reviewed ba7c086. Three issues need addressing before merge: shadow/advisory can suppress Memory creation, the declared privacy boundary does not constrain the actual backend call, and the Server logger drops the new observation fields. Reproductions and required behavior are in the inline comments.
Persist a Scope-isolated, reference-only observation ledger for Memory write-gate decisions. Sidecar failures do not alter gate outcomes.
Constraint: observations exclude candidate content, evidence bodies, model rationale, and exception messages.
Scope-risk: retention prunes expired observations only within the Scope receiving a write.
Tested: uv run --no-sync pytest tests/builtin/persistence/test_decision_observation_repository.py tests/builtin/runtime/test_decision_config.py tests/builtin/runtime/test_memory_write_gate_contract.py tests/builtin/runtime/test_memory_write_gate_paths.py; uv run --no-sync ruff check <16 changed files>; uv run --no-sync ruff format --check <16 changed files>; uv run --no-sync ty check --python-version 3.11 <16 changed files>.
Apply mode semantics to evidence preflight rejections, enforce loopback-only local evaluation, and make privacy outcomes observable in the private decision ledger and JSON logs.
Constraint: observations retain bounded references and counts only; no candidate or evidence content is logged or stored.
Scope-risk: unknown, LAN, wildcard, and hosted backends fail open under local_only until an explicitly loopback endpoint is configured.
Tested: focused gate, policy, composition, logging, and MySQL schema suite (108 passed); Ruff, format, ty 3.11, and diff check.
The reason will be displayed to describe this comment to others. Learn more.
Re-reviewed 62b4117 and replayed the original counterexamples: the shadow/advisory, privacy-boundary, and serialized-log findings are fixed. Two remaining issues in the new observation ledger need addressing; details and reproductions are inline.
Clarify that replay and rescore are promotion evidence for enforcing policies, rather than mandatory runtime capabilities for every consumer. Track the Memory write-gate evidence work in #1930.
Constraint: the RFC does not require additional consumers, a dashboard, replay tooling, or a cross-consumer evaluation bundle.
Scope-risk: consumers without promotion evidence remain disabled, shadow, or advisory; existing authority, Scope, revision, and citation boundaries are unchanged.
Tested: website pnpm lint; website pnpm test (13 passed); website pnpm build (255 public pages and 405 repository files link-validated; 950 public pages export-verified); git diff --check.
Adds governed decision-policy contracts and applies them to the Memory write gate with privacy modes, structured observations, and durable persistence.
Changes:
Introduces decision policy, assessment, privacy, and observation types.
Adds mode-aware Memory gating and scoped observation retention.
Updates schema, recovery docs, logging, and focused tests.
ConfigDict(frozen=True) does not make this nested mapping immutable; callers can mutate the underlying dictionary after policy creation. assess_decision_result() will then execute a different yes/no mapping under the same policy ID and version, violating the versioned-policy invariant. Store the mapping in a genuinely immutable representation or defensively freeze it at validation/use boundaries.
Treat disabled mode as inactive before backend resolution
memory_write_gate_enabled=True with the newly supported mode=disabled is treated as a missing-backend failure: the builder returns None, after which composition logs “no decision backend is available” even when one exists. The surrounding pipeline checks also still resolve/register the decision backend solely from memory_write_gate_enabled. Treat disabled mode as an inactive consumer before backend resolution and suppress the unavailable warning for this valid configuration.
Log observations only after final action is determined
This emits the observation before Memory knows the final domain action. For example, an accepted assessment is logged as memory_write_accept, then _prepare_commit() can hit the capacity error covered by the new test and the durable sidecar is finalized as memory_write_failed; the structured log permanently reports a final action that never occurred. Defer this observation log until the service finalizes held/unchanged/failed/applied, or mark this event as provisional and omit final_action.
The reason will be displayed to describe this comment to others. Learn more.
Reviewed cd2bc83. Requesting changes: local_only can send raw content through a non-loopback redirect, and successful file-backed SQLite ingestion loses its durable observation after waiting on its own transaction. The two new inline comments cover the SQLite transaction and Entry-identity failures; privacy, policy consistency, batch references, and final-action logging are updated in their existing threads with current reproductions. Six distinct findings remain (2 P1, 4 P2).
The reason will be displayed to describe this comment to others. Learn more.
[P1] Finalize observations without a competing SQLite writer
On cd2bc83, public Source capture followed by memory.flush() on real file-backed SQLite writes Memory revision 1 and advances the cursor to 1, but leaves zero ledger observations. This sink opens a separate write transaction while RelationalMemoryIngestion.flush() still owns the transaction bound to MemoryService.apply(). The insert waits for the default 5-second busy timeout, raises OperationalError, and is swallowed as decision.observation_sink_failed; the reproduced flush took 5.61 seconds. Finalize at the actual owner transaction boundary so accepted ingestion writes retain their observations without waiting on their own uncommitted writer.
The reason will be displayed to describe this comment to others. Learn more.
Fixed in abeed49c. Atomic window publish now appends the finalized observation through the same connection that commits Atomic Memory and advances the Source cursor. There is no independent RelationalDecisionObservationSink writer during the owning SQLite transaction. The file-backed Atomic window regression verifies a durable finalized observation after the write.
The reason will be displayed to describe this comment to others. Learn more.
[P2] Encode supported Entry identities before persisting references
Source/Artifact components are now encoded, but the applied subject refs still insert raw Entry IDs. Using the public RelationalContexts composition with a valid ASCII id_factory returning entry/1 and version/1 successfully writes and reads Memory, yet this emits entry:entry/1@version/1. The repository allowlist rejects it with ValueError; the sink failure is swallowed and the ledger contains zero observations. Memory's existing validate_identifier() permits these bounded ASCII IDs. Encode Entry and version components reversibly and unambiguously too, so audit coverage does not depend on their spelling.
The reason will be displayed to describe this comment to others. Learn more.
Fixed as part of the Atomic Memory migration in abeed49c. The removed collection Entry/Version identities are no longer used as sidecar subjects. Atomic publish records final Artifact references, with family and artifact-id components encoded using URL-safe base64 before repository validation, so supported delimiters and Unicode in generated IDs cannot make the audit insert disappear.
Finalize gate observations only after the Memory owner knows the domain
outcome, bind relational sidecars to a caller-owned transaction, and retain
encoded entry references for every assessed candidate in an applied batch.
Reject supplied policies that the specialized gate cannot execute literally,
and disable redirects for loopback OpenAI endpoints.
Constraint: Preserve fail-open gate behavior and avoid retaining raw candidate or evidence content.
Scope-risk: Changes observation finalization, relational transaction ownership, and local-only provider transport only.
Tested: pytest focused decision and memory gate suites; ruff check/format; ty check --python-version 3.11.
Rebase the decision-policy slice onto Atomic Memory and finalize each
observation in the source-window owner transaction after the domain action.
Loopback OpenAI transports refuse redirects before raw decision content can
leave the local endpoint.
Constraint: preserve Atomic Memory authority, source-window cursor semantics, and content-free audit sidecars.
Scope-risk: the legacy collection Memory service remains deleted; the gate is applied only to Atomic Memory ingestion.
Tested: uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_paths.py tests/builtin/runtime/test_family_processing.py tests/builtin/runtime/test_memory_window_recovery.py tests/builtin/runtime/test_processing_composition.py tests/builtin/runtime/test_decision_composition.py tests/builtin/runtime/test_decision_config.py tests/builtin/runtime/test_memory_write_gate_contract.py tests/builtin/persistence/test_decision_observation_repository.py -q
The reason will be displayed to describe this comment to others. Learn more.
Re-reviewed abeed49c. Source-ID encoding and successful Atomic sidecar persistence are fixed. Remaining redirect and final-log issues are updated in their existing threads; the additional shadow-mode failure is inline.
CI also fails in test_wizard_skill_selection_accepts_dream_derivation[enable-skill], reproduced locally: its model_copy override leaves generation_base_url as a string, while the new loopback check accesses .host. The fixture needs a validated URL, or that boundary needs to normalize the input.
Defer terminal decision events until the Atomic Memory transaction commits, preserve shadow writes when audit persistence fails, and disable redirects for custom Anthropic header clients.
Constraint: audit writes stay in the owner transaction and malformed audit writes never block the domain write.
Scope-risk: changes are limited to decision observation finalization and custom provider redirect transport.
Tested: uv run --no-sync pytest tests/builtin/runtime/test_decision_composition.py tests/builtin/runtime/test_memory_window_recovery.py -q; uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_paths.py tests/builtin/runtime/test_processing_composition.py tests/builtin/runtime/test_memory_write_gate_contract.py -q; ruff and ty checks passed.
The reason will be displayed to describe this comment to others. Learn more.
Re-reviewed 992a0f07. The Anthropic redirect, ordinary audit-failure, and rollback-log counterexamples are fixed. A transaction-aborting audit error still loses Memory while consuming its Source window; that P1 reproduction is in the existing audit-failure thread. The additional diagnostic issue is inline.
Allow ordinary observation persistence failures to degrade without blocking the owner write, but propagate SQLite savepoint loss caused by a full owner rollback. Log only the bounded exception class for degraded audit failures.
Constraint: Audit writes remain in the owner transaction and no diagnostic contains database exception payloads.
Scope-risk: SQLite RAISE(ROLLBACK) now fails the flush rather than committing a later cursor update after the owner write was lost.
Tested: uv run --no-sync pytest tests/builtin/runtime/test_memory_window_recovery.py tests/builtin/runtime/test_memory_write_gate_paths.py tests/builtin/runtime/test_processing_composition.py tests/builtin/runtime/test_memory_write_gate_contract.py -q; uv run --no-sync ruff check src/powercontext/builtin/runtime/atomic_memory_processing.py tests/builtin/runtime/test_memory_window_recovery.py; uv run --no-sync ruff format --check src/powercontext/builtin/runtime/atomic_memory_processing.py tests/builtin/runtime/test_memory_window_recovery.py; uv run --no-sync ty check --python-version 3.11 src/powercontext/builtin/runtime/atomic_memory_processing.py tests/builtin/runtime/test_memory_window_recovery.py; git diff --check
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
hidb4ai
left a comment
knqiufan
left a comment
Which issue or RFC does this PR close? Implements the first bounded Memory write-gate slice of RFC #1770. Refs #1770, #1742, and #1745. This PR does not close the broader implementation or evaluation issues. # Rationale for this change RFC #1770 requires an explicit policy, privacy, fallback, observation, and promotion contract before additional domain consumers adopt decision models. Memory write evidence sufficiency is the narrowest existing boundary: the gate may produce a visible
HOLD, but Memory retains authority over commits and revisions. The design borrows only bounded governance principles from Hermes/JEV research: explicit modes, deterministic local rules, safe fallback, and observable outcomes. It does not copy Hermes storage, lifecycle, prompts, thresholds, or host-specific implementation. # What changes are included in this PR? - Adds runtime-independentDecisionPolicy,DecisionAssessment, andDecisionObservationcontracts. A fallback is alwaysunadjudicated. - Adds an opt-in Memory evidence-sufficiency gate withdisabled -> shadow -> advisory -> enforcingbehavior. Runtime configuration defaults toshadow; enforcing is an explicit operator choice. - Binds the gate's hold direction and confidence threshold into the effective immutable policy version, and declares labeled calibration, measured friction/cost/latency, and rollback as promotion criteria. - Applies mode semantics and a private observation to deterministic evidence-projection rejections as well as model decisions. Existing custom gates without the optional preflight contract retain conservative legacy behavior. - Enforces the privacy boundary on every gate construction path. Raw candidate/evidence text is evaluated only whenlocal_onlyis selected and the resolved backend is explicitly loopback (localhost,127.0.0.1, or::1). Unknown, LAN, wildcard, and hosted endpoints fail open without receiving raw content.no_external_call,references_only, and unsupportedhosted_redacteddo not make a content call. - Emits bounded structured logs and a private, scope-isolated durablepc_decision_observationsledger. Observations retain policy/version/mode, stable references, counts, verdict/action, privacy boundary/outcome, and fallback codes only; they never store candidate text, evidence text, model rationale, or exception details. - Adds exact observation replay idempotency, same-ID conflict rejection, and optionaldecision_observation_retention_dayscleanup limited to the current Scope. The default is no automatic deletion. - Completes MySQL schema/index budgeting and bilingualobloaderrecovery lists for the new table. # Are there any user-facing changes? No public HTTP/OpenAPI endpoint, CLI command, dashboard, export surface, hosted-redacted transport, replay/rescore job, query rewrite, or additional decision-model consumer is added. The Memory write gate remains disabled unless explicitly enabled. When enabled, its runtime default isshadowand its privacy default isno_external_call; it only records a no-call observation and preserves the write. An operator must explicitly selectlocal_only, configure a loopback backend, calibrate the policy, and chooseenforcingbefore raw content is evaluated and a write can be held. # How was this change tested? -uv run --no-sync pytest tests/builtin/runtime/test_memory_write_gate_contract.py tests/builtin/runtime/test_memory_write_gate_paths.py tests/builtin/runtime/test_decision_config.py tests/builtin/runtime/test_decision_composition.py tests/builtin/runtime/test_decision_policy.py tests/test_server_logging.py tests/builtin/persistence/test_decision_observation_repository.py tests/builtin/persistence/test_mysql_schema.py -q(129 passed) -uv run --no-sync ruff check <16 changed files>-uv run --no-sync ruff format --check <16 changed files>-uv run --no-sync ty check --python-version 3.11 <14 changed Python files>-git diff --checkValidation limitation: the broadertests/builtin/runtime -qrun consistently stops in the unchanged Windows spawned-worker diagnostic case expectingworker_crashbut observingprocessing_failed(197 passed, 1 failedbefore--maxfail=1). This PR does not modify that worker or diagnostic path; the focused decision, persistence, logging, and composition suites above were run after the final changes. # AI usage statement Implemented with OpenAI Codex in the Codex desktop app from RFC #1770 and repository-local inspection. Hermes/JEV was consulted only as architectural prior art; no Hermes source or implementation structure was copied.Follow-up
Promotion to
enforcingis not part of this PR. The replay/re-evaluation, calibration, held-out evaluation, and promotion decision needed to consider it are tracked in #1930.